Audio to video synchronization

Audio to video synchronization (also known as audio video sync, audio/video sync, AV-sync, lip sync, or by the lack of it: lip sync error, lip-flap) refers to the relative timing of audio (sound) and video (image) parts during creation, post-production (mixing), transmission, reception and play-back processing. When sound and video have a timing related cause and effect, AV-sync can be an issue in television, videoconferencing, or film.

Digital or analog audio video streams or video files usually contains some sort of explicit AV-sync timing, either in the form of interlaced video and audio data or by explicit relative time-stamping of data. The processing of data must respect the relative data timing by e.g. stretching between or interpolation of received data. If the processing does not respect the AV-sync error, it will increase whenever data gets lost, because of transmission errors or because of missing or mis-timed processing.

Contents

Incorrectly synchronized

There are different ways in which the AV-sync can get incorrectly synchronized:

Examples of transmission (broadcasting), reception and playback that can get the AV-sync incorrectly synchronized:

Recommendations

For television applications, audio should lead video by no more than 15 milliseconds and audio should lag video by no more than 45 milliseconds.[1] For film, acceptable lip sync is considered to be no more than 22 milliseconds in either direction.[2]

MPEG: Presentation Time Stamp (PTS), Decode Time Stamp (DTS)

Presentation time stamps (PTS) can be embedded in MPEG transport stream to avoid AV-sync drift. Unfortunately these time stamps are often added after the video undergoes frame synchronization, format conversion and pre-processing, thus those delays remain uncompensated.[3][4][5][6]

Viewer experience of incorrectly synchronized AV-sync

The result typically leaves a filmed or televised character moving his or her mouth when there is no spoken dialog to accompany it, hence the term "lip flap" or "lip-sync error". The resulting audio video sync error can be annoying to the viewer and can even lead to the viewer's not enjoying the program, to the program's not being effective, and to the speakers being perceived negatively.[7] The lack of effectiveness problems are of particular concern when product commercials and political candidates are viewed. Television industry standards organizations, such as the Advanced Television Systems Committee, have become involved in setting standards for audio video sync errors.[1]

Because of these annoyances, AV-sync error is of concern to the television programming industry, including television stations, networks, advertisers and program production companies. Unfortunately the advent of high definition flat panel display technologies (LCD, DLP and plasma) which can delay video more than audio have moved the problem into the viewer's home and beyond control of the television programming industry alone. Consumer products companies now offer audio delay adjustments to compensate for video delay changes in TV's, a/v receivers, and several companies manufacture dedicated digital audio delays made exclusively for lip-sync error correction.

Effect of no explicit AV-sync timing

When a digital or analog audio video stream does not have some sort of explicit AV-sync timing these effects will cause the stream to become out of sync:

Sources/references

  1. ^ a b c IS-191: Relative Timing of Sound and Vision for Broadcast Operations, ATSC, 2003-06-26, archived from the original on 2011-07-27, http://www.webcitation.org/60UbU5Ziv 
  2. ^ Sara Kudrle et. al. (July 2011). "Fingerprinting for Solving A/V Synchronization Issues within Broadcast Environments". Motion Imaging Journal (SMPTE). "Appropriate A/V sync limits have been established and the range that is considered acceptable for film is +/- 22 ms. The range for video, according to the ATSC, is up to 15 ms lead time and about 45 ms lag time" 
  3. ^ MPEG-2 Systems FAQ: 19. Where are the PTSs and DTSs inserted?
  4. ^ MPlayer-G2-dev: mpeg container's timing (PTS values)
  5. ^ birds-eye.net: DTS - Decode Time Stamp
  6. ^ svcd2dvd.com: Perfect AV Sync: Preparation is key...
  7. ^ Byron Reeves; David Voelker (October 1993). "Effects of Audio-Video Asynchrony on Viewer's Memory, Evaluation of Content and Detection Ability" (PDF). http://www.pixelinstruments.tv/pdf/Articles/Effects%20of%20Audio-Video%20Asynchrony.PDF. Retrieved 2008-10-19. 
  8. ^ "The relative timing of the sound and vision components of a television signal" (PDF). http://tech.ebu.ch/docs/r/r037.pdf. 

See also

External links